Power-law distribution functions derived from maximum entropy 
and a symmetry relationship 
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Power-law distributions are common, particularly in social physics. Here, we explore whether 
power-laws might arise as a consequence of a general variational principle for stochastic processes. 
We describe communities of 'social particles', where the cost of adding a particle to the community 
is shared equally between the particle joining the cluster and the particles that are already members 
of the cluster. Power-law probability distributions of community sizes arise as a natural consequence 
of the maximization of entropy, subject to this 'equal cost sharing' rule. We also explore a general- 
ization in which there is unequal sharing of the costs of joining a community. Distributions change 
smoothly from exponential to power-law as a function of a sharing-inequality quantity. This work 
gives an interpretation of power-law distributions in terms of shared costs. 
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Power-law probability distributions are ubiquitous in 
nature, especially in social systems. For example, the 
fraction pk of U.S. cities with a population of k peo- 
ple scales as pk ~ fc~^-^^ [TH3]. Other examples of 
power-law distributions include incomes [4], Internet 
links [5l[5], fluctuations in stock market prices [7HS], com- 
pany sizes [TU] , numbers of citations received by scientific 
papers [TTVfO] . and many others [3]. 

Exponential distributions, such as the Boltzmann dis- 
tribution law, are also ubiquitous. But, whereas ex- 
ponential distributions have a well-known basis in the 
principles of statistical physics [T4Ul6j . it is unclear if a 
similar principle underlies power-law distributions. Here, 
our interest is in how power-law distributions might arise 
from stochastic processes, particularly in social physics. 
Our approach is based on the principle of maximum en- 
tropy (MaxEnt). MaxEnt is widely used, not only in 
thermal physics, but also in image analysis [TTHTO] . draw- 
ing inferences [20l [21] , and in nonequilibrium statistical 
mechanics [52H11]- We show here that the same prin- 
ciple, with a 'cost-sharing' type of constraint, leads to 
power-law distributions. 

Clustering can be framed in terms of 'joining 
costs'. We focus on a problem of particle clustering, 
which provides a convenient language for comparing peo- 
ple joining cities to the statistical physics of growing col- 
loids and polymers. We first describe a standard growth 
mechanism, which we express in terms of the 'joining 
costs' of a particle to a growing cluster. One particle may 
stick to another, forming a cluster of size 2. Another par- 
ticle may join the cluster, forming a size 3 cluster, and so 
on. If there are N total particles, then the equilibrium 



of clusters of various sizes can be written as 

1 + 1^2 
1 + 2^3 



(1) 



1 + (A^-1) 



N 



Let rik be the number of clusters of size k at equilib- 
rium. 



Uk = Kuk-iUi = K'^ ^nin2 ■ ■ ■ nfc-i 



(2) 



where K is an equilibrium binding constant. The prob- 
ability distribution, pk , of cluster sizes is the ratio of the 
number of size k clusters to the total number of clusters 
of all sizes, 



Pk = Q ^nk, (3) 
where Q is the grand canonical partition function, 

N fc-1 

Q = T. ^'^'li^r (4) 

fc=i j=i 



Statistical physics provides an alternative language for 
expressing the logarithms of populations in terms of ener- 
gies, free energies or chemical potentials, which are cost- 
like additive quantities. Here, we define a dimensionless 
chemical-potential- like quantity, fik , that we call the join- 
ing cost for a particle to join a size-fc cluster. 



Mfc = ln(i4:nfc) . 
Re-expressing Q in terms of these costs gives 



Ek-i 
7 = 1 f^3 — 



(5) 
(6) 
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where 



Wk 



fc-1 



(7) 



2 



is the dimensionless cost of assembling the whole cluster. 
In this language, if Wk is positive, the distribution will 
be dominated by small clusters; if Wk is negative, the 
system will preferentially populate large clusters. We use 
this cost language for social physics below. To proceed 
further, we need to know how Wk depends on k. 

Independent particle clusters are exponentially 
distributed. First, for illustration, we treat a standard 
problem of colloid assembly or polymerization. Assume 
the cost /ife for a particle to join a cluster is independent 
of A;. Then 



(8) 



where the constant /i° is the cost of adding one particle 
to a cluster of any size. A cluster of size k requires k — 
1 particle additions, so the total cost of assembling the 
cluster is 



Wk = /i° (fc - 1) , 



(9) 



and the average cost of adding a particle, taken over all 
cluster sizes, is 



(w) 



k 



WkPk = M° ((fc) - 1) . 



(10) 



To predict the probability distribution of cluster sizes, 
we maximize the entropy. 



S = - ^Pk Inpfc 



(11) 



subject to two constraints: (1) a fixed known value of (w) 
(Eq. 10 ) and (2) normalization (ensuring the probabilities 
sum to 1), 



Y^Pk 



1. 



A constraint on (w) gives the extremum function 
(w) - S = p° ^(fc - l)pk +y^Pfclnpfc, 

k k 

so that the optimization condition is 

Y,dpl[\upl + l + a + p° (fc-1)] =0, 



(12) 



(13) 



(14) 



where a and n° are the Lagrange multipliers that enforce 
normalization and constraint |10[ respectively. Solving 



Eq. 14 gives the equilibrium probability distribution, p^, 
which maximizes the entropy and satisfies the average 
cost and normalization constraints. The solution is the 
standard exponential distribution of cluster sizes. 



Pfe=e ' "e - - =(^'le-^°^ (15) 
where Q is the grand canonical partition function. 



N 
k=l 



fi°k _ 



1 



(16) 



Equal cost sharing leads to povirer-lavir distri- 
butions. Now, we consider a more general notion of a 
particle's joining cost when it enters a fc-mer cluster. A 
person is more likely to join a larger city than a smaller 
city because of greater opportunities of jobs, infrastruc- 
ture, services, entertainment, and other economic and 
social factors. Existing citizens have already paid some 
of the cost of entry for the new joiner 'particle'. So, rela- 
tive to the cost of joining an independent-particle cluster 
(/i°), the cost fik of joining a 'social-particle' cluster of 
size k is reduced to 



fJ-k = P- 



kvk- 



(17) 



Eq. 17 expresses the idea of cost sharing, namely that the 
cost of joining a cluster is reduced because the existing 
k member particles provide a discount of each to the 
joiner particle. 

There are two non-arbitrary limiting cases for how we 
might choose the value of r^: (1) No sharing, = 0, 
and the particles are independent, as described above, or 
(2) Equal sharing, where each member pays the same 
amount as the joiner when it enters the cluster, 

Tk = fJ-k- (18) 
Substituting Eq. [18] into [TT] and solving for fi^ yields 



1 + fc' 



(19) 



which expresses how the costs diminish with cluster size 
in social-particle systems. The total cost to assemble a 
social cluster of k particles is 



fe-i 
o \ ' 1 



i=i 



which can be expressed as (see Appendix) 

1 



In k 



7-1 



(20) 



(21) 



where 7 = 0.5772... is Euler's constant. The average 
joining cost per particle is 



In k 



1 



7-1 



(22) 



To predict the social-particle probability distribution, 

on (w). 



In ( A: + i ) + 7 - 1 



we maximize the entropy subject to constraint 22 
^dpl I Inpl + 1 + a + /i 

Solving for p^. yields 

' k 



Pk 



w 0. 

(23) 

(24) 
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giving a power-law distribution. The scaling exponent 
/i° is the 'un-discounted cost' of adding one particle to a 
cluster Q 

The present work shows how power-law distributions 
can emerge naturally from random clustering of particles 
that equally share the joining cost. The 'rich-get-richer' 
aspect of power-law size distributions is that social parti- 
cles are more attracted to bigger clusters than to smaller 
clusters. We have expressed this in a language of 'costs': 
the power-law arises because member particles equally 
share the joining costs with joiner particles. The power- 
law exponent is the 'un-reduced' cost 

The aim of the present work is not to build a model 
of a specific power-law process. We do not give here a 
generative way to derive fi° for any particular problem of 
social physics. Rather, the present treatment plays a role 
more like the Boltzmann law, which is a framework for 
models, rather than a particular model itself. There is a 
large literature of models that generate power laws [551 
We believe that, at least in some cases, such models 
may be interpretable in terms of shared costs. 

In many situations involving people, the idea of cost 
sharing arises naturally. For example, a well-studied 
power-law distribution is of the number of citations to 
scientific papers [TltiTS] . Suppose the author of paper A 
is preparing to cite paper C. Every paper in the set {B} 
that has already cited paper C can be thought of as a 
member of a community that paper A is about to join. 
The larger is the community {B}, the more likely it is 
for paper A to join that community, and cite paper C. 
The community {B} has already paid a higher cost by 
finding paper C in sea of options that was larger at the 
time. In this way, any paper B has lowered the cost for 
the author of paper A to find and cite paper C. 

The distribution of U.S. city sizes also has a power- 
law tail lU [2j ■ It is more likely a person will choose to 
move to Los Angeles, CA (population: 9 million) than 
to Fields, Oregon (population: 86). Similar to the cost 
amortization obtained through an economy of scale, the 
people of LA have already paid the development costs of 
creating the companies, jobs, infrastructure, and services 
that attract new individuals. Thus, the marginal cost of 
adding one more person is reduced, relative to the cost 
of adding a person to a smaller city. 

A generalized model for partial cost sharing. 
Described above are two limiting cases: no cost sharing 
(independent particles) or full cost sharing (the members 
pay the same as the joiner). What if the member particles 



This scaling form for p^. \24h is relatively accurate for most values 
of fc (A: > 2). The exact forms, valid for all values of k, are given 
in the Appendix. 

Our term 'costs' here can alternatively be thought of in terms 
of 'economies of scale'. Making more widgets decreases the cost- 
per- widget. That is, if the cost of a widget factory is fi° , then 
the cost per widget for the first widget is ii° and the cost per 
widget for making two widgets is /^°/2. In this sense, the cost of 
the second is 'shared' by the first. 
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FIG. 1. Probability distributions for various sharing values, 
with fi° = 2. At s = 0.1, the particles are barely helpful 
at all, and the entering particle must pay most of its joining 
cost, so the distribution is nearly exponential. At s = 0.9, the 
particles are very helpful, and the distribution is a power-law. 



only pay a fraction s of the cost that the joiner particle 
pays? Now the cost reduction is 

Tfe = s^ik- (25) 

Combining Eq. [25] with Eq. |17| and solving for /i^, we find 



1 



(26) 



so the total cost of assembling a partially-social cluster 
of size k is 



k-l 



const, (27) 



where the constant will be absorbed into the normaliza- 
tion (see Appendix). Maximizing the entropy gives 



Pk 



1 1 



(28) 



We call s the sharing parameter: s — Q involves no 
sharing (i.e., independent particles) and s = 1 involves 
full equal sharing between the joiner particle and all 
members. The s parameter controls the shape of the 
distribution. For s = 0, the entry cost is the same for 
all clusters, and the distribution is exponential; there are 
very few large clusters. For s = 1, the particles are social, 
and the entry cost is reduced for larger clusters, resulting 
in a power-law distribution; there are more large clusters 
in this case. Fig. [T] shows that varying s changes the dis- 
tribution smoothly from exponential to power-law. We 
refer to the limit s — oo as 'super-social particles': the 



4 



existing members pay the full cost, and the joiner parti- 
cle pays nothing. For super-social particles, we obtain a 
uniform distribution; all cluster sizes are equally proba- 
ble. 

We have shown how power-law distributions can arise 
naturally from the maximization of entropy subject to 
a symmetry relationship in which all particles share the 
cost incurred when a new particle joins a cluster. It has 
been noted before that MaxEnt with logarithmic con- 
straints leads to power-law distributions [27l [28] |^ Ex- 
ponential distributions result from constraints on linear 
averages such as (fc), while power-law distributions result 
from constraints on logarithmic averages such as (In k) . 
Here, we have described how such constraints can be in- 
terpreted as a type symmetry of sharing that is natural 
in the social realm. 

Many previous studies on generative mechanisms for 



power-laws have investigated a family of 'proportional 
attachment' (PA) rules [m l30ti32] . In the context of 
particle clustering, a PA rule says that the probability 
of a cluster acquiring a new particle is proportional to 
the number of particles it already contains. A premise 
of the PA rule is that the joining particle is cognizant of 
the populations of the clusters in the system. By con- 
trast, our cost-sharing framework does not assume the 
particles know anything about the system, so the present 
approach may be useful for modeling systems composed 
of 'uninformed' particles. 
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Logarithmic constraints have also been justified in terms of the 
information content of language |29l , to explain the observation 
that word- use frequency obeys a power-law [T] I26| . 
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APPENDIX 

Approximate cost function for social particles. 

Here, we derive expression 21 by including order l/fc cor- 
rections from the sum-to-integral conversion of Wk for so- 
cial particles. We use the Euler-Maclaurin formula [33] 



to convert the sum in Eq. 19 to an integral: 

k-l 

~ + j 



X:^^lnfc+l- + C + 0(.-) 



(29) 



where C is an unknown constant. 

First, we calculate the asymptotic form of \n{k + a) 
(for some constant a, where A: ^ a) by Taylor expanding 
around a = 0: 



ln{k + a) Ink + - + O (k-^) 



(30) 



We use 30 to absorb the 1/k term in 29 into the loga- 
rithm, 



lnfc+A-^ln(fc+i 



(31) 



Next, we evaluate the constant C. We begin by defining 
the k'^^ harmonic number, 



k 

Hk = y^^ = ^{i+k) 



+ 7, 



(32) 



where i^{k) = o?lnr(fc)/o?fc is the digamma function, and 
7 — 0.5772... is Euler's constant. The sum in Eq. [I9| can 

be written 



k-l ^ 

E = iffe - 1 = 1^ (1 + fc) + 7 - 1. 



(33) 



Ignoring the term in Eq. [33] that depends on fc, we see 
that the remaining constant terms are 



C = 7-l. 



(34) 



Substituting 31 and 34 into 29 we find 

fc-i 



^ i-f 



In ( fc + i 
+ J V 2 



-l-l + Oik- 



(35) 



Dropping the order l/k"^ corrections gives the expression 
used in [21] 

Exact distributions. In Eq. [2lj we have given an 
approximation to pk ■ Here, we use Eq. [33] to obtain the 
exact expression for the probability distribution function 
Pk- We write the cost function Wk as 



Wk = IJ- 



fc-i 



1 



= M 



i/j (1 + fc) + 7 - 1 



^1 + 7 

Maximizing the entropy yields 



p^ = g-ie-^>(i+'=), 

with partition function 



N 

fc=i 



(36) 



(37) 



(38) 



In the more general case of partially social particles, 
the cost function is 



k-l 



1 



sj s 



tp[^+k 



1 



leading to the probability distribution 

pl = Q-'e-^^ii+'^), 

where 

N 



Q = ^e-^^(H'=). 



(39) 
(40) 

(41) 



fc=i 



